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This  development  was  conducted  in  support  of  Navy  Decision  Coordinating  Paper, 
Z0109-PN  under  subproject  Z0109-PN.03  (Manpower  Cost  in  System  Design)  and  was 
sponsored  by  the  Deputy  Chief  of  Naval  Operations  (OP-01).  The  subproject  concerns  the 
application  of  human  engineering  technology  in  the  development  of  procedures  to 
incorporate  hardware/software/personnel  trade-offs  and  cost  benefit  alternatives  in  all 
stages  of  system  design. 

The  present  development  was  undertaken  in  1978  to  gain  insight  into  how  engineers 
conceptualize  skills  required  of  personnel  who  operate  and  maintain  the  hardware  systems 
being  designed  for  future  Navy  use.  Information  obtained  was  used  to  prepare  a  draft 
version  of  an  engineer's  guide  entitled  Designing  for  Human  Skills  in  Navy  Electronic 
Systems  (NPRDC  TN  81-15).  Further  development  of  this  guide  was  abandoned  in  favor  of 
a  related  guide  published  as  An  Engineer's  Guide  to  the  Use  of  Human  Resources  in 
Electronic  Systems  Design  (NPRDC  TN  79-8)  and  an  evaluation  of  that  guide  (NPRDC  SR 
81-3).  Information  obtained  through  the  earlier  engineers'  concepts  effort  is  being 
documented  at  this  time  for  distribution  to  the  research  community. 

The  contracting  officer's  technical  representative  was  Mr.  Ernest  A.  Koehler. 


SUMMARY 


Progga 


In  the  design  process  for  new  hardware,  selection  of  a  particular  configuration 
influences  such  factors  as  the  number,  skills,  and  training  needs  of  operating  and 
maintenance  personnel  required  to  support  the  system.  These,  in  turn,  significantly 
influence  manpower  life  cycle  costs.  Despite  the  importance  of  these  equipment- 
personnel  trade-offs,  in  which  skill  is  a  primary  factor,  the  hardware  developer  often  has 
little  understanding  of  the  impact  his  decisions  will  have  on  personnel  factors. 

Objectives 

— - ^This  effort  was  conducted  as  part  of  a  program  aimed  at  developing  tools  for  the 
hardware  developers  to  use  In  assessing  the  personnel  implications  and  costs  of  alternative 


design  options. 


The  objectives  of  this  effort  were  to  determine 


ine  JW^the  kinds  of 


skill  concepts 


engineers  apply  to  their  designs  and  £tiPwhether  the  sophistication  of  these  skill  concepts 


can  be  increased  by  presenting  the  engineer  with  a  structured  framework  based  on 


behavioral  research. 


A  battery  of  tests  was  developed  and  administered  to  a  representative  group  of  40 
design  engineers.  The  procedure  required  engineers  to  estimate  skill  levels  required  by 
operation  and  maintenance  tasks.  In  an  unstructured  survey,  the  engineers  were  asked  to 
list  the  most  important  tasks  to  be  performed  in  the  operation  or  maintenance  of 
equipment  on  which  they  had  recent  design  experience.  For  each  combination  of  any 
three  tasks  listed,  the  engineers  were  asked  to  indicate  the  name  of  the  skill  required  by 
any  two  of  these  tasks  and  the  degree  of  skill  required  by  each. 

In  a  structured  survey,  the  engineers  were  given  two  sets  of  cards.  One  set  included 
29  cards,  15  of  which  described  an  operation  task,  and  the  other  14,  a  maintenance  task. 
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The  other  set  Included  22  cards,  11  of  which  described  a  cognitive  skill,  and  the  other  11, 
a  psychomotor  skill.  For  each  of  these  skills,  engineers  were  to  sort  the  task  description 
cards  into  five  skill  levels;  that  is,  they  had  to  determine  whether  the  task  required  none, 
a  small  amount,  a  moderate  amount,  a  high  degree,  or  a  maximum  amount  of  the  skill  in 
question. 

Conchnions 

1.  Engineers  have  relatively  few  and  nondifferentiated  concepts  of  operation  and 
maintenance  skills. 

2.  They  consider  that  the  skills  required  for  operating  and  maintenance  tasks  differ 
significantly.  Equipment  maintenance  is  more  difficult,  requiring  a  higher  level  of  skill 
that  is  oriented  primarily  on  cognitive  capabilities,  whereas  operating  tasks  require  only 
psychomotor  abilities. 

3.  It  is  possible  to  increase  the  sophistication  of  the  engineers'  skill  concepts  by 
providing  them  with  a  structured  situation  that  leads  them  through  the  skill-analysis 
process. 


Itlvi  uli'I  [’Ll 


A  personnel  design  requirements  handbook  should  be  developed  to  enable  design 
engineers  to  assess  the  personnel  implications  of  hardware  system  design  concepts. 
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In  the  design  process  for  new  hardware,  selection  of  a  particular  configuration 
influences  such  factors  as  the  number,  skills,  and  training  needs  of  operating  and 
maintenance  personnel  required  to  support  the  system.  These,  in  turn,  significantly 
influence  manpower  life  cycle  costs.  Despite  the  importance  of  these  equipment- 
personnel  trade-offs,  in  which  skill  is  a  primary  factor,  the  hardware  developer  often  has 
little  understanding  of  the  impact  his  decisions  will  have  on  personnel  factors. 

Objectives 

This  effort  was  conducted  as  part  of  a  program  aimed  at  developing  tools  for  the 
hardware  developers  to  use  In  assessing  the  personnel  implications  and  costs  of  alternative 
design  options. 

The  objectives  of  this  effort  were  to  determine  (1)  the  kinds  of  skill  concepts 
engineers  apply  to  their  designs  and  (2)  whether  the  sophistication  of  these  skill  concepts 
can  be  increased  by  presenting  the  engineer  with  a  structured  framework  based  on 
behavioral  research. 
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A  battery  of  tests  was  developed  and  administered  to  a  representative  group  of  40 
design  engineers.  The  procedure  required  engineers  to  estimate  skill  levels  required  by 
operation  and  maintenance  tasks.  In  an  unstructured  survey,  the  engineers  were  asked  to 
list  the  most  important  tasks  to  be  performed  in  the  operation  or  maintenance  of 
equipment  on  which  they  had  recent  design  experience.  For  each  combination  of  any 
three  tasks  listed,  the  engineers  were  asked  to  indicate  the  name  of  the  skill  required  by 
any  two  of  these  tasks  and  the  degree  of  skill  required  by  each. 

In  a  structured  survey,  the  engineers  were  given  two  sets  of  cards.  One  set  included 
29  cards,  19  of  which  described  an  operation  task,  and  the  other  14,  a  maintenance  task. 
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The  other  set  included  22  cards,  11  of  which  described  a  cognitive  skill,  and  the  other  11, 
a  psychomotor  skill.  For  each  of  these  skills,  engineers  were  to  sort  the  task  description 
cards  into  five  akill  levels;  that  is,  they  had  to  determine  whether  the  task  required  none, 
a  small  amount,  a  moderate  amount,  a  high  degree,  or  a  maximum  amount  of  the  skill  in 
question. 

Conctoions 

1.  Engineers  have  relatively  few  and  nondifferentiated  concepts  of  operation  and 
maintenance  skills. 

2.  They  consider  that  the  skills  required  for  operating  and  maintenance  tasks  differ 
significantly.  Equipment  maintenance  is  more  difficult,  requiring  a  higher  level  of  skill 
that  is  oriented  primarily  on  cognitive  capabilities,  whereas  operating  tasks  require  only 
psychomotor  abilities. 

3.  It  is  possible  to  increase  the  sophistication  of  the  engineers'  skill  concepts  by 
provking  them  with  a  structured  situation  that  leads  them  through  the  skill-analysis 

A  personnel  design  requirements  handbook  should  be  developed  to  enable  design 
engineers  to  assess  the  personnel  implications  of  hardware  system  design  concepts. 
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INTRODUCTION 


Problem  and  Background 


This  development  is  part  of  an  effort  to  optimize  the  manpower  required  to  operate 
and  maintain  Navy  systems.  Personnel  costs  are  the  major  part  of  life  cycle  costs.  For 
example,  personnel  costs  represent  about  61  percent  of  the  total  FY78  DoD  budget  (King, 
1977),  and  personnel  costs  account  for  about  58  percent  of  the  annual  operating  expense 
for  each  DD  963  class  destroyer  (Director  of  Defense  Research  and  Engineering,  1973). 

The  higher  the  skill  level  demanded  by  Navy  equipment,  the  more  costly  the 
personnel  who  must  operate  and  maintain  that  equipment.  This  is  because  these  skills 
must  be  provided  by  personnel  who  have  completed  Navy  training,  and  the  greater  the 
skill  demanded,  the  longer  and  more  expensive  the  courses  required  to  train  those  skills. 
Therefore,  a  primary  requirement  in  attempting  to  reduce  manpower  costs  is  to  reduce 
the  skills  needed  to  utilize  Navy  equipment. 

It  is  obvious  that  the  design  of  Navy  equipment  determines  manning  in  general,  and 
personnel  skill  level  in  particular.  Taylor  (1975)  indicated  that  70  percent  of  the  life 
cycle  costs  of  new  systems  are  determined  by  decisions  made  in  the  concept  phase  of 
hardware  development.  If  manpower  needs  and  their  associated  costs  are  to  be  reduced, 
the  nature  of  equipment  design  must  be  influenced  early,  so  that  fewer  skilled  personnel 
are  required. 

Is  this  possible?  Although  technological  advances  and  increased  technological 
sophistication  work  against  this,  the  fact  remains  that  alternative  equipment  designs  are 
possible  for  any  system  requirement.  In  other  words,  to  satisfy  the  same  requirement  one 
can,  for  example,  design  systems  that  are  completely  automated,  partially  automated,  or 
largely  manual.  Some  of  these  designs  demand  personnel  with  lower  skill  levels  than 
others.  All  other  things  being  equal,  if  the  designers  are  aware  of  two  alternative 
designs— one  demanding  a  lower  skill  level  than  another— they  are  more  likely  (or  at  least 
it  is  a  logical  expectation)  to  select  the  alternative  calling  for  fewer  personnel  with  lower 


skills.  This  is  the  thinking  represented  by  contract  specifications  calling  for  the  simplest 
possible  design  tailored  to  the  lowest  possible  skill  levels. 

Design  engineers  are  at  the  center  of  efforts  to  reduce  manpower,  because  they  are 
largely  responsible  for  the  equipment  configuration  that  ultimately  reaches  the  Navy.  It 
is,  therefore,  necessary  to  determine  whether  engineers  are  aware  of  skill  factors  and 
their  relationships  to  design  characteristics.  If  it  is  difficult  for  them  to  differentiate 
between  alternative  configurations  on  the  basis  of  the  required  skills,  they  cannot  make 
design  trade-offs  involving  skills  and  select  the  most  cost-efficient  design  alternative. 

It  is  essential  to  investigate  the  engineers'  "conceptual  space"  so  that  means  can  be 
found  of  influencing  their  design  perceptions.  Unfortunately,  relatively  little  is  known 
about  engineers  despite  their  central  role  in  system  development.  Previous  studies 
(Meister,  Sullivan,  &  Askren,  1968)  Askren  &  Lintz,  1975)  have  demonstrated  the 
feasibility  of  studying  the  engineer  as  an  element  in  design,  but  much  more  remains  to  be 
learned. 

Objectives 

This  study  was  conducted  as  part  of  a  program  aimed  at  developing  tools  for  the 
hardware  developers  to  use  in  assessing  the  personnel  implications  and  costs  of  alternative 
design  options. 

The  objectives  of  this  effort  were  to  determine  (1)  die  kinds  of  skill  concepts 
engineers  apply  to  their  designs  and  (2)  whether  the  sophistication  of  these  skill  concepts 
can  be  increased  by  presenting  the  engineer  with  a  structured  framework  based  on 
behavioral  research. 


METHOD 

Subjects 

In  any  study  in  which  the  design  process  is  at  issue,  the  selection  of  the  subject 
sample  is  extremely  important.  Unfortunately  precise  criteria  for  differentiating 
between  different  types  of  engineers  ~e>  ..  At  the  very  least,  however,  one  should 


be  concerned  with:  (1)  level  of  design  experience  (system  or  major  assembly),  (2) 
knowledge  of  operation  or  maintenance  functions,  and  (3)  type  of  system  or  major 
assembly  designed. 

Table  1  shows  the  characteristics  of  the  subject  engineers.  Thirty-two  subjects  were 
selected  from  Hughes  Aircraft  Corporation  and  eight  from  the  Autonetics  Division  of 
Rockwell  International. 


Table  1 

Characteristics  of  the  Subject  Sample 


Type  of 

System 

Operation  Tasks 

Maintenance  Tasks 

System 

Level 

Major  Assembly 
Level 

System 

Level 

Major  Assembly 
Level 

Command/ control 

2 

2 

2 

2 

Communications 

2 

2 

2 

2 

Fire  control 

2 

2 

2 

2 

Surveillance 

2 

2 

2 

2 

Autonetics 

2 

2 

2 

2 

Liuujects'  ages  ranged  from  24  to  57  years,  with  a  median  of  43  years.  Their  design 
experience  ranged  from  1  to  36  years,  with  a  median  of  15  years.  The  percentage  of 
years  in  a  supervisory  role  ranged  from  0  to  100  percent,  with  a  median  of  10  percent. 


The  distribution  of  education,  as  indicated  by  the  highest  year  of  formal  schooling 
completed,  is  shown  in  Table  2. 

The  major  subject  areas  of  academic  training  reported  (at  some  time  during  formal 
education  program)  included  business  administration  (1),  computer  science  (4),  economics 

(1) ,  electronics  (4),  engineering  (3),  electrical  engineering  (19),  mechanical  engineering 

(2) ,  systems  engineering  (2),  general  science  (2),  management  (1),  mathematics  (4), 
physics  (5),  and  psychology  (1).  The  total  number  exceeds  40  because  several  persons 
worked  in  different  major  subject  areas  during  undergraduate  school  than  in  graduate 
school. 
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Table  2 


Distribution  of  Subject  Education 


Highest  Year 

No.  of  Subjects 

12 

1 

13 

2 

14 

4 

15 

1 

16 

12  (median) 

17 

8 

18 

9 

19 

2 

20 

1 

* 


Recent  design  experience  was  reported  on  these  Navy  electronic  systems:  Surface 
Towed  Array  Sonar  System,  Surface  Sonar  System  Modernization  Program,  Improved 
Point  Defense  Target  Acquisition  System,  Electro-Static  Gyro  Monitor,  AN/UYQ-21 
Computer  Display  Set,  Navy  Tactical  Data  System,  Position  Locating  and  Reporting 
System,  DD  963  Electronics  System,  DD  963  Exterior  Communications  System,  Secure 
Voice  System,  Anti-Ship  Torpedo  Defense  System,  AN/SPS-52  Radar  System,  AN/PRC- 
104  Radio,  JTIDS/TDMA  "B"  Terminals,  CV-TSC  (ASW  Sonar),  Tunable  Attributes 
System,  Fast-Time  Analyzer  System,  Shipboard  Data  Multiplex  System,  Aided  Display 
Submarine  Control  System,  and  Passive  Sonar  Processing  System. 

Motivation  was  excellent  in  all  cases,  except  for  two  subjects  who  were  judged  to  be 
neutral.  The  most  frequent  comment  was  that  most  system  design  efforts  did  not 
remotely  approach  the  level  of  detail  on  task  and  skill  requirements  evident  in  the  test 
battery.  Several  supervisory  engineers  requested  copies  of  the  task  and  skills  lists  to  use 
in  checklist  fashion  in  future  proposals  or  system  designs. 
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Test  Procedure 


1 


The  procedure  employed  required  engineers  to  estimate  skill  levels  required  by 
operation  or  maintenance  tasks  which  they  generated. 

In  the  unstructured  survey,  the  engineers  were  asked  to  assume  that  they  were 
currently  involved  in  the  design  of  equipment  on  which  they  had  recently  worked.  (The 
instruments  used  are  presented  in  the  appendix.)  The  following  steps  were  performed  for 
operation  and  maintenance  tasks: 

1.  They  were  asked  to  list  the  most  important  tasks  to  be  performed  for  operation 
or  maintenance  of  that  equipment. 

2.  For  selected  combinations  of  the  tasks  listed  in  step  1,  taken  three  at  a  time, 
they  were  asked  to  indicate  (in  their  own  words)  the  skill  required  by  any  two  of  these 
tasks. 

3.  Having  now  specified  the  skills  required  by  these  tasks,  the  subjects  were  asked 
to  indicate  the  degree  of  that  skill  required  by  each  of  the  tasks  listed  in  step  1. 

In  the  structured  survey  (the  instructions  are  in  the  appendix),  the  following  steps 
were  performed: 

1.  The  subjects  were  given  a  set  of  29  cards,  one  for  each  of  the  15  operation  task 
descriptions  and  the  14  maintenance  task  descriptions  (shown  in  Table  3).  These  task 
descriptions  were  developed  after  review  of  task  taxonomies  previously  developed  by 
Askren  (1976);  Berliner,  Angell,  and  Shearer  (1974);  Finley,  Obermayer,  Bertone,  Meister, 
and  Muckier  (1970);  Parker  (1975);  and  Wylie,  Dick,  and  Mackie  (1975). 

2.  The  subjects  were  also  given  a  set  of  22  cards,  one  of  each  of  1 1  cognitive  skills 
and  11  psychomotor  skills.  The  skills  taxonomy  was  based  largely  on  the  factor  analytic 
research  of  Fleishman  (1972),  as  modified  by  Ekstrom  (1973)  and,  particularly,  Dunnette 
(1976).  The  skills  are  also  listed  in  Table  3. 
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Tabic  3 


Task  and  Skill  Descriptions 


Title 


Description 


Task  Descriptions 


Operation] 

I.  Initiate  equipment  operation 


2.  Establish  desired  operating  modes 


3.  Perform  correct  sequence 
of  operating  procedures 


4.  Observe  and  interpret  visual 
displays  and  indicators 


5.  Recognize  and  interpret 
auditory  signals 


£.  Read  and  understand  text 


7.  Operate  discrete 
control  devices 


I.  Operate  continuous 
control  devices 


9.  Interpret  visual  and 

auditory  data  to  assist  in  decision 
making 

10.  Perform  quantitative 
computations 


1 1.  Select  appropriate  course 
of  action 


12.  Supply  or  enter  data  to 
implement  decisions 

19.  Receive/transmit  communications 
relevant  to  operation 


14.  Monitor  equipment  operation 


13.  Perform  preventive  maintenance 


Maintenance! 

1.  Obtain  physical  see  ess 
to  the  equipment 

2.  Select  and  use  tools  necessary 
lor  maintenance 


3.  Perform  preventive  maintenance 


Set  up  and  initialise  equipment/system 
in  accordance  with  established  proce¬ 
dures. 

Select  and  set  up  specific  operating 
modes  in  response  to  established  criter¬ 
ia,  or  in  response  to  changing  operating 
conditions.  ■* 

Operate  equipment/system  in  accord¬ 
ance  with  established  procedures  and  the 
requirements  of  command  and/or  the  en¬ 
vironment. 

Scan,  detect,  identify,  extract  features 
of,  and  process  the  absence  or  presence 
of  in- tolerance  or  out-of-tolerance  data 
presented  via  equipment/system  visual 
displays  and  indicators. 

Detect,  identify,  extract  features  of, 
and  process  the  absence  or  presence  of 
in-tolerance  or  out-of-tolerance  condi¬ 
tions  indicated  by  auditory  signals. 

Read,  Interpret,  and  extract  information 
from  textual  material  (e.g.,  written, 
printed,  or  displayed). 

Select  and  make  correct  use  of  discrete 
controls  such  as  switches,  selectors,  and 
keyboards. 

Select  and  make  correct  use  of  continu¬ 
ous  control  devices  such  as  control 
sticks,  trackballs,  verniers,  and  wheels. 

Identify  relevant  visual  and  auditory  sig¬ 
nals,  extract  pertinent  values,  or  data, 
and  process  them  for  use  in  making 
equipment/system  operating  decisions. 

Using  established  logic  and  procedures 
and  available  tools,  execute  required 
computations  utilizing  quantitative  in¬ 
formation. 

Based  on  available  data,  command  in¬ 
struction,  precedent,  and  established 
procedures,  determine  a  course  of  action 
matched  to  the  mission  and  capabilities 
of  the  equipment/system. 

Operate  equipment/system  devices  to 
make  data  inputs  that  identify  and  im¬ 
plement  the  course  of  action  selected. 

Select  appropriate  mode  of  communica¬ 
tion  and  receive  or  transmit  data  regard¬ 
ing  equipment/system  status,  decisions, 
or  future  operation. 

Locate,  identify,  and  interpret  equip¬ 
ment/system  status  indicators  as  requir¬ 
ed  to  ascertain  that  satisfactory  operat¬ 
ing  conditions  are  being  maintained. 

In  accordance  with  established  proce¬ 
dures,  carry  out  the  preventive  actions 
required  to  maintain  the  equipment/sys¬ 
tem  in  satisfactory  operating  condition. 


Locate  equipment  requiring  mainten¬ 
ance,  approach  it  safely,  and  open/re¬ 
move  the  necessary  access  panels. 

Recognize,  acquire,  and  use  correctly 
the  tools  appropriate  to  the  maintenance 
tasks  to  be  performed. 

Perform  any  and  all  tasks  required  on  a 
periodic  basis  to  maintain  equipment  in 
"  operational  condition  (e.g,,  cleaning,  oil¬ 
ing,  adjusting,  removing,  and  replacing 
filters  etc.). 


Table  3  (Continued) 


Description 


Task  Descriptions 


Maintenance  (Con  t.h 

4.  Operate  BITE,  and/or  connect 
and  operate  support  or  test 
equipment 

5.  Set  initial  conditions  required 
lor  maintenance 

6.  Inspect  equipment 


7.  Isolate  malfunction  to 
identifiable  unit 


8.  Verily  failure  or  malfunction 


9.  Perform  in-place  adaptive 
corrections 

10.  Perform  corrosion  control 
procedures 

1 1.  Remove  malfunctioning  unit/ 
component 

12.  Replace  malfunctioning  unit/ 
component 


1 3.  Verify  satisfactory  functioning 
of  replacement  unit/component 


14.  Restore  equipment/system  to 
operational  condition 


Select  support  or  test  equipment  requir¬ 
ed  for  maintenance,  connect/hook  it  up 
properly,  and  operate  it  correctly  and 
safely. 

Set  up  equipment/system  in  accordance 
with  established  maintenance/operation 
procedures. 

Conduct  a  physical  inspection  of  the 
equipment/system  with  regard  to  main¬ 
tenance  needs. 

Using  appropriate  procedures,  BITE, 
and/or  test  equipment  and  tools,  locate 
or  trace  out-of-tolerance  conditions  or 
malfunctions  to  discrete  or  identifiable 
unit(s)  or  components). 

Using  procedures,  BITE,  and/or  test 
equipment  and  toots,  verify  the  initial 
diagnosis  as  to  the  location  and  nature  of 
the  failure  or  malfunction. 

If  corrective  action  can  be  accomplished 
in-place,  use  procedures,  tools  and  test 
equipment  to  do  so. 

Select  and  apply  appropriate  corrosion 
control  action  based  upon  equipment 
type,  location,  and  status. 

Isolate  and  physically  remove  defective 
unit/component  from  the  equipment/sys¬ 
tem  being  served. 

Acquire  an  operational  unit/component 
and  use  appropriate  procedures,  tools, 
and  test  equipment  to  install  it  as  a 
replacement  for  the  malfunctioning 
unit/component. 

Using  established  procedures,  too|s,  and 
test  equipment,  verify  that  the  replace¬ 
ment  unit/component  is  functioning  to 
specified  tolerance  levels. 

Conduct  necessary  operating  checks,  re¬ 
move  all  tools,  test  and  support  equip¬ 
ment,  close  up  the  equipment/system, 
clean  up  area,  and  return  equipment/sys¬ 
tem  control  to  operational  personnel. 


Skill  Descriptions 


Cognitive: 

1.  Pattern  sensitivity 


2.  Fluency 


3.  Idea  iormation 


4.  Rote  memory 


Ability  to  distinguish  form  or  pattern 
within  a  confusing  background,  such  as 
detecting  a  target  or  symbol  in  display 
clutter,  or  locating  a  particular  ob|ect  or 
tool  in  a  box  containing  an  assortment  of 
such  items. 

Ability  to  find  ways  of  saying  things  that 
are  most  appropriate  to  particular  situa¬ 
tions,  ideas,  or  concepts,  such  as  sum¬ 
marizing  developments  in  a  tactical  situ¬ 
ation,  or  reporting  and  describing  the 
nature  of  an  unexpected  malfunction. 

Ability  to  find  and  test  ideas  that  show 
how  many  things  are  interrelated  within 
a  larger  system,  such  as  making  an  as¬ 
sessment  of  a  tactical  situation,  or  sum¬ 
ming  up  the  nature  ol  a  system-level 
malfunction. 

Ability  to  memorize  related  or  apparent¬ 
ly  unrelated  items  and  to  recall  most  or 
ail  of  the  memorized  information  when 
presented  with  only  an  element  or  part 
of  it,  such  as  routine  operational  proce¬ 
dures,  or  diagnostic  steps  in  fault  isola¬ 
tion  tasks. 


Table  3  (Continued) 


Title 


Description 

Skill  Deacriptiona 


Cognitive  (Con  t.k 
3.  Span  memory 


6.  Number  facility 


7.  Viaual  speed 


t.  Deductive  reasoning 


9.  Spatial  orientation 


10b  Spatial  visualisation 


II.  Verbal  comprehension 


Ability  to  recall  perfectly  for  immediate 
reproduction  a  series  of  items  after  only 
one  presentation  of  the  scries,  such  as 
repeating  flight  data  during  air  traffic 
control  communications,  or  relaying 
sparepart  nomenclature  during  urgent 
repair/replacement  work. 

Ability  to  add,  subtract,  multiply,  and 
divide  rapidly  with  few  errors,  such  as 
performing  arithmetic  operations  involv¬ 
ing  date-time  indications,  display  track 
headings,  test  instrument  readings,  or 
control  settings. 

Ability  to  detect  visual  signals  quickly, 
make  comparisons,  and  carry  out  other 
simple  operations  involving  visual  per¬ 
ception,  such  as  detection  of  visual  alert 
displays,  or  awareness  of  changes  in 
equipment  status  indicators. 

Ability  to  reason  from  given  data  to  the 
necessary  conclusion,  such  as  deciding  on 
a  course  of  action  based  on  tactical 
displays  and  communications,  or  deter¬ 
mining  the  nature  of  a  malfunction  based 
on  Indications  during  troubleshooting. 

Ability  to  tell  where  subject  is  in  rela¬ 
tion  to  some  object,  or  to  tall  where  the 
object  it  in  relation  to  subject,  such  as 
awareness  of  compass  directions  linking 
subject's  location  with  the  locations  of 
related  military  units,  or  awareness  of 
the  location  of  particular  equipments 
within  a  complex  installation. 

Ability  to  visualise  forms  and  patterns  in 
the  Imagination,  and  to  move  them  about 
or  change  them  mentally,  such  as  ability 
to  anticipate  near-futive  tactical  devel¬ 
opments  from  present  dtepiays,  or  to  see 
hoar  to  fit  a  component  into  an  odd- 
shaped  apace. 

Knowledge  of  wards  and  their  meaning, 
as  wall  as  the  application  of  this  know! 
edge  to  the  understanding  of  verbal  com¬ 
munication,  such  as  rapid  and  accurate 
understanding  of  operational  orders  and 
instructions,  or  technical  manual 
passages  end  diagnostic  task 
descriptions. 


I.  Control  precisian 


2.  Multiiimb  coordinating 


3.  Response  orientation 


novement 


Ability  to  perform  finely  controlled  mus¬ 
cular  adjustments,  such  as  moving  a 
lever  to  a  precise  setting,  or  setting  of 
vernier  knob  to  an  exact  position. 

Ability  to  coordinate  the  movements  of 
arms  and/or  legs  simultaneously,  such  as 
in  climbing  a  ladder,  operating  a  type¬ 
writer,  or  pecking  a  number  of  equip¬ 
ment  items  into  a  box. 

Ability  to  make  correct  and  accurate 
movements  In  relation  to  a  stimulus 
under  highly  speeded  conditions,  such  as 
reaching  out  and  flicking  a  switch  when 
a  warning  horn  sowids,  or  resetting  the 
correct  circuit  breaker  in  a  panel  con¬ 
taining  an  array  of  circuit  breakers. 

Ability  to  respond  rapidly  when  a  stimu¬ 
lus  occurs,  such  as  answering  an  incom¬ 
ing  telephone  or  radio  call,  or  shutting 
down  equipment  when  inoperable  condi¬ 
tions  are  Indicated. 

Ability  to  make  rapid  arm  movements 
where  accuracy  is  not  required,  such  as 
gathering  trash  or  debris  snd  throwing  it 
into  a  large  pile. 


*.  Reaction  time 


Table  3  (Continued) 


Title 


Description 


Psychomotor  (Con  t.k 
6.  Hate  control 


7.  Manual  dexterity 


8.  Finger  dexterity 


9.  Arm-hand  steadiness 


10.  Wrist-finger  speed 


1 1.  Aiming 


Skill  Descriptions 


Ability  to  make  continuous  equipment 
adjustments  relative  to  a  moving  target 
changing  in  speed  and  direction,  such  as 
following  a  bird  with  a  rifle,  or  tracking 
a  target  across  a  CRT  display  using  a 
trackball  or  light  pen. 

Ability  to  make  skiiiful  arm  and  hand 
movements  in  handling  rather  large  ob¬ 
jects  under  speeded  conditions,  such  as 
assembling  or  disassembling  a  military 
rifle. 

Ability  to  make  skillful  manipulations  of 
small  objects  with  the  fingers,  such  as 
sorting  out  an  assortment  of  objects,  or 
making  nut-and-bolt  connections. 

Ability  to  make  precise  arm-hand  posi¬ 
tioning  movements  that  do  not  require 
strength  or  speed,  such  as  routine  solder¬ 
ing  of  wires,  or  replacement  of  delicate 
equipment  components  or  subassemblies. 

Ability  to  make  rapid  tapping  move¬ 
ments  with  the  wrist  and  fingers,  such  as 
transmitting  Morse  Code  with  a 
telegraphic  key,  or  tapping  equipment 
components  with  a  small  hammer  to 
assure  secure  positioning. 

Ability  to  activate  small  equipment  ele¬ 
ments  quickly,  repeatedly,  and 
accurately,  such  as  data  entry 
keyboards,  test  point  probes,  or 
communication  channel  twitches. 


3.  The  engineers  were  then  asked  to  take  the  top  card  from  their  skill  decks  and 
place  it  in  the  top  center  bin  of  a  sorting  board.  The  particular  skill  title— and 
definition- -on  that  card  was  the  skill  to  be  applied  throughout  the  first  sort.  The  next 
five  cards  in  the  deck  carried  skill  level  indicators;  these  were  placed  in  the  center  row  of 
the  sorting  board  in  sequence  from  left  to  right  (level  //I  on  the  extreme  left  bin,  level  U2 
next,  etc.  as  shown  in  Figure  1).  The  five  skill  level  cards  represented  a  5-point  Likert 
scale,  as  defined  below: 

Skill  level  1:  The  task  does  not  require  this  skill  at  all. 

Skill  level  2:  The  task  requires  a  small  amount  of  this  skill. 

Skill  level  3:  The  task  requires  a  moderate  amount  of  this  skill. 

Skill  level  4:  The  task  requires  a  high  degree  of  this  skill. 

Skill  level  5:  The  task  requires  a  maximum  amount  of  this  skill. 


/Skill  Title 
& 

Definition 


wt 

fl 

■ 

mmrni 

■E  mm 

■ 

ns  in 

■ 

■ 

iffin 

mgm 

M 

"s  I 

11 

if 

a 

1 

nwwm 

B 

Bp  p  Bn 

m 

BPBB!  !I 

m 

ITWITB1, 

Figure  1.  Structured  survey:  Card  sorting  procedure. 
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4.  The  remaining  cards  in  the  first  card  deck  were  the  task  titles  and  descriptions 
shown  in  Table  3.  Taking  each  task  card  in  succession,  the  subject  read  it,  noted  the  skill 
title  and  definition  in  the  top  center  bin,  and  estimated  the  level  of  skill  (1  through  5) 
required  to  perform  that  task.  The  task  card  was  then  placed  in  the  appropriate  bin  in  the 
bottom  row  of  the  sorting  board,  just  below  the  skill  level  card  carrying  the  number  of  the 
estimated  skill  level. 

5.  The  engineer  then  continued  through  the  package  of  task  cards,  sorting  them  as 
in  step  4.  When  one  package  of  task  titles  was  completed,  the  experimenter  picked  up  the 
cards  and  the  subject  proceeded  to  the  next  package  of  cards  (i.e.,  a  second  skill  title  with 
definition  and  the  13  operation  and  14  maintenance  tasks). 

RESULTS  AND  DISCUSSION 
Responses  to  the  Unstructured  Survey 

Since  engineers  were  not  given  a  formal  framework  for  reporting  the  tasks  and  skills 
in  the  unstructured  survey,  it  was  impossible  to  pool  calculations  across  subjects.  Some 
observations  were  made: 

1.  The  96  different  action  or  key  words  used  by  the  subjects  to  describe  the  tasks 
and  skills  required  in  system  operation  and  maintenance  are  listed  in  alphabetical  order  in 
Table  4.  In  a  few  cases,  the  same  word  was  termed  both  a  task  and  a  skill  required  for 
both  operation  and  maintenance.  In  other  cases,  the  same  word  appeared  in  two  or  three 
categories.  Most  listings  were  generated  in  only  one  of  the  four  categories. 
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Table  4 


Task  and  Skill  Keywords  Used  by  Engineers 


Operator  Tasks 

Maintenance  Tasks 

Operator  Skills 

Maintenance  Skills 

1 

— 

Ability 

Ability 

2 

Adjust 

Adjust 

— 

— 

3 

Alert 

— 

— 

— 

4 

Align 

Align 

— 

— 

5 

Analyze 

Analyze 

— 

Analyze 

6 

— 

— 

— 

Aptitude 

7 

— 

Assemble 

— 

— 

S 

Assign 

—  ' 

— 

— 

9 

— 

— 

Association 

— 

10 

— 

Bring  up 

— 

— 

11 

— 

— 

Capability 

— 

12 

Catalog 

— 

— 

— 

13 

Check 

Check 

— 

— 

14 

Classify 

— 

— 

— 

15 

— 

Clean 

— 

— 

16 

Communicate 

— 

— 

— 

17 

— 

Compare 

Compare 

— 

IS 

— 

— 

Concentrate 

— 

19 

Conduct 

— 

— 

— 

20 

Control 

— 

— 

— 

21 

Coordinate 

— 

Coordinate 

— 

22 

— 

Correct 

— 

— 

23 

— 

— 

Correlate 

— 

24 

Decide 

— 

— 

25 

— 

— 

— 

Deduce 

26 

Designate 

— 

— 

— 

27 

— 

— 

— 

Desire 

2S 

Detect 

— 

Detect 

— 

29 

Determine 

— 

— 

30 

— 

— 

Dexterity 

Dexterity 

31 

Discriminate 

— 

Discriminate 

— 

32 

Edit 

— 

— 

— 

33 

Enter  Data 

— 

— 

— 

34 

Evaluate 

Evaluate 

Evaluate 

Evaluate 

35 

— 

— 

Experience 

— 

36 

Extract 

— 

Extract 

— 

37 

Eye-hand  Coord. 

— 

Eye-hand  Coord. 

— 

3S 

— 

— 

Familiarity 

Familiarity 

39 

Follow  Instructions 

Follow  Instructions 

Follow  Instructions 

Follow  Instructions 

40 

— 

— 

Gather  Information 

Gather  Information 

41 

Handle 

— 

— 

— 

42 

Identify 

Identify 

Identify 

— 

43 

Initialize 

Initialize 

— 

— 

44 

— 

Install 

— 

— 

45 

Interpret 

Interpret 

Interpret 

Interpret 

46 

— 

— - 

— 

Intuition 

47 

— 

Isolate 

— 

— 
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Table  4  (Continued) 


Operator  Tasks  Maintenance  Tasks  Operator  Skills  Maintenance  Skills 


48  - 

— 

Judgment 

— 

49  Knowledge 

— 

Knowledge 

Knowledge 

50  Localize 

Localize 

— 

— 

51  - 

Lubricate 

— 

_ 

52  - 

— 

Manipulate 

Manipulate 

53  - 

— 

Memory 

Memory 

54  - 

— 

— 

Measure 

55  Monitor 

Monitor 

— 

— 

56  - 

— 

— 

Motor  Skills 

57  Observe 

Observe 

— 

— 

58  - 

Obtain  (Retrieve) 

— 

— 

59  ~ 

Operate 

Operate 

Operate 

60  - 

— 

Perception 

— 

61  Perform 

Perform 

— 

62  - 

•  - 

— 

Practice 

63  Prepare 

— 

— 

— 

64  - 

— 

— 

Problem  Solve 

65  Program 

— 

Program 

— 

66  React 

— 

— 

— 

67  - 

Read 

Read 

Read 

68  - 

— 

— 

Reasoning 

69  Recognize 

Recognize 

Recognize 

Recognize 

70  - 

~ 

Relate 

— - 

71  - 

Remove 

— 

Remove 

72  - 

Repair 

— 

Repair 

73  - 

Replace 

— 

Replace 

74  Report 

Report 

— 

75  - 

— 

Respond 

— 

76  - 

Restore 

— 

— 

77  - 

Run 

— 

78  Search 

— 

— 

— 

79  Select 

Select 

Select 

80  Set  up 

Set  up 

— 

— 

81  - 

— 

— 

Solder 

82  - 

— 

Spatial  Relations 

— 

83  Supervise 

— 

— 

— 

84  Test 

Test 

— 

— 

85  - 

— 

— 

Trace  Signals 

86  Track 

— 

Track 

— 

87  Troubleshoot 

Troubleshoot 

— 

Troubleshoot 

88  - 

Tune 

— 

89  - 

— 

Type 

Type 

90  - 

— 

Understand 

— 

91  - 

— 

Use 

Use 

92  - 

Verify 

— 

— 

93  ~ 

— 

Vision 

— 

94  - 

— 

— 

Visual  Activity 

95  ~ 

— 

Visual  Discrim. 

— 

96  - 

— 

Visualization 

Visualization 

2.  Despite  the  variety  of  terms  generated,  few  tasks  and  skills  were  listed.  The 
number  of  tasks  ranged  from  4  to  10,  with  a  median  of  7,  and  skills  ranged  from  2  to  16, 
with  a  median  of  5.  When  the  skill  level  estimates  were  subjected  to  factor  analysis,  the 
distribution  of  factors  was: 


Number  of  Factors  Frequency  (Ei 


4  2 

3  8 

2  16  (median) 

1  4 

0  10 


Thus,  engineers'  unstructured  responses  revealed  a  lack  of  standardized  vocabulary 
for  describing  tasks  and  skills,  and  relatively  simplistic  (few-factor)  concepts  of  operation 
and  maintenance  in  Navy  electronic  systems. 

These  results,  while  not  completely  unexpected,  are  somewhat  distressing.  To  make 
manpower  trade-offs  among  alternative  design  concepts,  engineers  must  analyze  their 
designs  in  terms  of  the  tasks  to  be  performed  by  each  alternative,  the  skills  these  tasks 
demand,  and,  in  particular,  the  levels  of  those  skills.  If  they  lack  the  capacities  to 
perform  such  analyses,  they  cannot  select  the  design  configuration  requiring  the  least 
skill.  It  is,  however,  not  surprising  that  they  lack  these  capabilities,  because  they  have 
not  been  trained  to  apply  behavioral  concept  structires  to  their  work.  Possibly,  if  they 
were  given  such  concept  structures,  they  would  exhibit  greater  fluency  in  performing 
task/skill  analyses. 

Responses  to  the  Structured  Task/Skill  Survey 

When  provided  with  the  structured  sets  of  14  maintenance  tasks,  13  operation  tasks, 
and  22  skills,  the  judgments  were  more  precise  and  concepts  of  operation  and  maintenance 
were  composed  of  more  factors  than  from  the  unstructured  survey. 

Engineers  were  able  to  differentiate  the  various  tasks  in  terms  of  a  3-point  scale  of 
skill  level,  which  they  were  incapable  of  doing  in  the  unstructured  survey.  This  suggests 
that  engineers  are  able  to  conceptualize  skill  level,  but  that  they  lack  a  language  or 
framework  for  expressing  these  concepts. 


For  example,  they  were  able  to  rank  the  various  operation  and  maintenance  tasks  in 
terms  of  the  types  and  levels  of  skills  demanded  by  these  two  task  categories  (Table  5).  It 
is  interesting  to  note  that  the  absolute  level  of  skill  required  by  maintenance  tasks  is 
considered  to  be  somewhat  higher  than  that  required  by  operations  tasks.  The  ranking  of 
the  individual  skills  per  task  type  also  varied,  suggesting  that  to  the  engineers  the  two 
types  of  tasks  demand  somewhat  different  patterns  of  skills. 


Table  5 

Skill  Requirements  as  Ranked  by  Engineers 
for  Two  Groups  of  Tasks 


Rank  Operation  Task  1 


Rote  memory 
Visual  speed 
Deductive  reasoning 
Idea  formation 
Verbal  comprehension 
Reaction  time 
Aiming 

Response  orientation 
Pattern  sensitivity 
Spatial  visualization 
Span  memory 
Spatial  orientation 
Finger  dexterity 
Number  facility 
Multilimb  coordination 
Fluency 

Control  precision 
Arm-hand  steadiness 
Wrist-finger  speed 

20  Rate  control 

21  Manual  dexterity 

22  Speed-of-arm  movement 


aRange  of  mean  ratings  on  5-point  scale:  2.88  to  1.47. 
^Range  of  mean  ratings  on  5-point  scale:  3.14  to  1.84. 


Rank 


Maintenance  Task 


Verbal  comprehension 
Rote  memory 
Finger  dexterity 
Multilimb  coordination 
Manual  dexterity 
Arm-hand  steadiness 
Control  precision 
Deductive  reasoning 
Visual  speed 
Aiming 

Pattern  sensitivity 
Idea  formation 
Spatial  visualization 
Spatial  orientation 
Span  memory 
Response  orientation 
Rate  control 
Number  facility 


Reaction  time 
Speed-of-arm  movement 
Wrist-finger  speed 
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Table  6  ranks  operation  tasks  by  difficulty  and  maintenance  tasks  in  terms  of 
engineers'  judgments  of  the  level  of  skill  required  for  task  performance.  This  table 
suggests  that  engineers  can  more  readily  differentiate  the  skill  level  requirements  for 
individual  maintenance  tasks  than  for  the  individual  operating  tasks.  The  maintenance 
task  rankings  appear  to  be  more  logical  than  the  corresponding  rankings  of  operating 
tasks.  For  example,  it  is  difficult  to  see  why  much  more  skill  is  required  to  operate 
discrete  control  devices  (3rd)  than  to  interpret  visual  and  auditory  data  (11th).  This  may 
also  indicate  that  engineers  consider  equipment  operation  a  much  more  homogeneous 
activity— and,  therefore,  more  difficult  to  fractionate— than  maintenance.  The  implica¬ 
tion  one  might  draw  from  this  is  that  engineers  would  find  it  easier  to  analyze  their  design 
configurations  in  terms  of  maintenance  skill  level  relationships  than  for  operating  tasks. 

Inasmuch  as  the  engineers  responded  to  the  structured  survey  in  terms  of  the  22  basic 
skills,  the  results  were  combined  and  a  single  factor  rotation  was  performed  for  the  entire 
body  of  data.  The  results  were  four  significant  factors  for  maintenance  skills  as 
conceptualized  by  our  sample,  and  three  significant  factors  for  the  operation  skills. 
Twelve  of  the  component  skills  were  considered  appropriate  to  the  maintenance  tasks;  and 
11  to  the  operation  task.  The  cognitive  skills- -such  as  idea  formation,  deductive 
reasoning,  and  number  facility— tended  to  appear  only  under  the  maintenance  designation, 
whereas  the  psychomotor  skills- -such  as  manual  dexterity  and  arm-hand  steadiness— were 
common  to  both.  It  would  seem  that  the  engineers  in  the  sample  conceive  of  maintenance 
as  a  more  demanding- -indeed,  more  skillful— process  than  operation. 


Table  6 


Task  Difficulty  Ranked  by  Engineers 


Rank 

Operation 

1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 
13 

Maintenance 

1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 


Teak 


Perform  correct  sequence  of  operating  procedures 

Supply  or  enter  data  to  implement  decisions 

Operate  discrete  control  devices 

Select  appropriate  course  of  action 

Receive/transmit  communications  relevant  to  operation 

Perform  preventive  maintenance 

Operate  continuous  control  devices 

Observe  and  interpret  visual  displays  and  indicators 

Establish  desired  operating  modes 

Monitor  equipment  operation 

Interpret  visual  and  auditory  data  to  assist  in  decision  making 

Initiate  equipment  operation 

Perform  quantitative  computations 

Read  and  understand  text 

Recognize  and  interpret  auditory  signals 

Perform  in-place  adaptive  corrections 

Isolate  malfunctions  to  identifiable  unit 

Verify  satisfactory  functioning  of  replacement  unit/component 

Verify  failure  or  malfunction 

Restore  equipment/system  to  operational  condition 

Operate  BITE,  and/or  connect  and  operate  support  or  test 
equipment 

Replace  malfunctioning  unit/component 

Remove  malfunctioning  unit/component 

Set  initial  conditions  required  for  maintenance 

Perform  preventive  maintenance 

Select  and  use  tools  necessary  for  maintenance 

Inspect  equipment 

Obtain  physical  access  to  equipment 
Perform  corrosion  control  procedures 


Note:  For  operation  tasks,  the  range  of  mean  ratings  on  a  3-point  scale  is  2.43  to  1.62 
/or  maintenance  tasks,  the  range  is  2.83  to  1.63. 


p 


N 


i. 
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Table  7  shows  the  four  dimensions  of  skill  generated  by  the  sample  for  maintenance, 
and  the  three  dimensions  generated  for  operation.  Only  two  of  these  were  unitary: 
Factor  IV  (reaction  time)  for  maintenance  and  Factor  III  (fluency)  for  operation.  Only 
those  components  having  factor  loadings  greater  than  the  square  root  of  2  are  shown, 
which  accounts  for  at  least  half  the  item  variance. 


Table  7 

Skill  Factor  Groupings  from  Structured  Survey 


Maintenance  Skill 

Operation  Skill 

Factor  I 

Factor  I 

Pattern  sensitivity 

Idea  formation 

Span  memory 

Number  facility 

Visual  speed 

Deductive  reasoning 

Control  precision 

Multilimb  coordination 
Manual  dexterity 

Finger  dexterity 

Arm-hand  steadiness 
Wrist-finger  speed 

Aiming 

Factor  II 

Factor  II 

Manual  dexterity 

Finger  dexterity 

Arm-hand  steadiness 

Pattern  sensitivity 

Spatial  orientation 

Spatial  visualization 

Factor  III 

Factor  III 

Spatial  orientation 

Speed-of-arm  movement 

Fluency 

Factor  IV 

Reaction  time 

Note:  Skill  groupings  from  factor  analysis.  Only  those  skills  are  shown  for  which  the 
associated  factor  accounted  for  more  than  half  the  variance  (loading  in  excess  of  /2). 


The  fact  that  6  of  the  12  appropriate  maintenance  skills  and  7  of  the  11  appropriate 
operation  skills,  respectively,  are  grouped  under  a  single  factor  indicates  that  these 
concepts  are  neither  well  attended  to  nor  well  differentiated  by  the  engineers  in  the 
sample.  This  conclusion  is  also  supported  by  the  small  number  of  factors  generated  for 
the  original  list  of  22  orthogonal  skill  components. 
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CONCLUSIONS 

1.  Engineers  have  relatively  few  and  nondifferentiated  concepts  of  operation  and 
maintenance  skills.  Their  responses  to  seif-generated  lists  of  tasks  and  skills  produced  a 
median  of  two  factors  or  skill  sets. 

2.  They  consider  that  the  skills  required  for  operating  and  maintenance  tasks  differ 
significantly.  Equipment  maintenance  is  more  difficult,  requiring  a  higher  skill  level  that 
is  oriented  primarily  on  cognitive  capabilities,  whereas  operating  tasks  are  oriented 
around  psychomotor  abilities. 

3.  It  is  possible  to  increase  the  sophistication  of  the  engineers'  skill  concepts  by 
providing  them  with  a  structured  situation  that  leads  them  through  the  skill  analysis 
process.  Their  responses  to  prescribed  lists  of  tasks  and  skills  produced  a  median  of  five 
factors  (compared  with  the  two  factors  produced  by  the  unstructured  survey). 

The  implications  of  these  findings  are  both  distressing  and  hopeful.  Anyone  who 
expects  engineers  on  their  own  to  make  design  trade-offs  involving  skill  will  be  distressed 
by  the  results.  It  is  unlikely  that,  without  being  pressed  contractually,  engineers  will 
pursue  such  analyses;  even  if  they  were  to  do  so,  the  products  of  these  analyses  would 
have  limited  usefulness. 

On  the  other  hand,  it  is  a  hopeful  sign  that  the  engineers  can  improve  their  analytic 
capabilities  by  following  procedures  established  by  behavioral  scientists.  Since  it  is 
impossible  to  provide  a  personal  human  factors  specialist  for  each  design  engineer,  a 
surrogate  must  be  provided.  This  can  be  some  sort  of  design  guide  that  provides  a  sample 
procedure  with  the  data  necessary  for  implementation. 

RECOMMENDATIONS 

A  personnel  design  requirements  handbook  should  be  developed  to  enable  hardware 
designers  to  assess  the  personnel  implications  of  hardware  system  design  concepts. 
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APPENDIX 

TEST/SURVEY  BATTERY  INSTRUMENTS 


This  appendix  contains  copies  of  materials  and  instructions  used 
in  the  test/survey  battery  administered  to  40  engineer  subjects 
participating  in  the  study.  The  completed  test/survey  battery  and 
other  data  are  at  NAVPER5RANDCEN. 
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EFFECT  OF  EQUIPMENT  DESICN  CHARACTERISTICS 
ON  USER  SKILL-LEVEL  REQUIREMENTS 


GENERAL  INFORMATION 


We  are  asking  you  and  a  number  of  engineers  to  take  part  in  a  study 
aimed  at  improving  the  ways  in  which  developers  of  military  systems  give 
consideration  to  manpower  problems  early  in  system  design.  Although  the 
study  is  funded  by  the  Navy,  it  addresses  a  problem  which  is  common  to  all 
U.S.  service?:  the  complexity  and  sophistication  of  military  electronic 
systems  appears  to  be  outrunning  the  supply  of  persons  who  are  able  (even 
after  training)  to  operate  and  maintain  them. 

The  particular  part  of  this  problem  we  are  studying  has  to  do  with 
the  differences  in  skill  level  that  might  be  required  of  Navy  crew  persons 
as  a  result  of  differences  in  equipment  items  chosen  by  an  engineer  when 
he  first  designs  an  electronic  system.  Ultimately,  the  Navy  is  seeking  an 
effective  way  to  help  engineers  take  the  skill-level  problem  into  account 
when  these  early  design  decisions  are  being  made.  Any  such  "effective  way" 
will  have  to  begin  with  engineers  like  yourself,  and  with  information  in¬ 
dicating  how  you  perceive  this  problem  and  how  you  make  design  decisions. 

The  long-range  objective  of  this  and  related  studies  is  to  discover 
how  various  design  characteristics  may  impact  the  skill  levels  required 
of  personnel  who  operate  and  maintain  the  resulting  equipment.  To  achieve 
the  objective  of  our  specific  study,  we  have  several  survey-type  booklets 
and  procedures  we  would  like  you  to  complete.  Some  concentrate  on  tasks 
performed  by  military  personnel,  and  the  skills  that  are  necessary  for 
accomplishing  those  tasks.  Another  will  examine  the  way  you  can  visualize 
a  simple  form  when  it  is  hidden  within  a  complex  pattern,  and  still  an¬ 
other  will  ask  how  you  usually  regard  certain  management/social/individual 
situations. 

With  the  exception  of  the  design  puzzle  booklet,  there  are  no  "right” 
or  "wrong"  answers  to  any  of  the  survey  items  you  will  encounter.  We  need 
your  thoughtful  responses  to  the  survey  items  solely  in  terms  of  your  en¬ 
gineering  experience  to  date,  and  your  present  knowledge  and  feelings. 


After  we  have  completed  our  analysis  of  the  response  data  you  will  be 
informed  of  the  results.  Our  plan  is  to  summarize  the  group  data  without 
identifying  individuals  by  name,  but  to  indicate  to  each  of  you  separately 
the  location  of  his  or  her  responses  with  respect  to  the  grouped  data. 


RESPONDENT'S  BIOGRAPHICAL  DATA 


Your  Name  . . Your  Age  . 

Undergraduate  Education: 

Years  Completed . Major  Field  . 

Craduate  Education: 

Years  Completed. .  Major  Field  . 

Years  of  Design  Experience . 'X  in  Management/Supervisory  Role.. 

e 

Name  of  Most  Recent  System  Worked  On  . 

Nature  of  Your  Design  Work  on  the  System . . . 

Have  You  Worked  on  Similar  Systems  in  the  Past? 

Name  of  System  Nature  of  Design  Work 
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Unstructured  Survey 


ESTIMATES  OF  SKILL  LEVELS 
REQUIRED  BY  OPERATOR  TASKS 


This  survey  Is  part  of  a  study  attempting  to  Identify  the  Influence  exerted  on  a  system's 
engineering  design  by  the  deslgaer's  perception  of  the  skills  of  the  persons  who  will  operate 

the  system. 

Ob  these  pages  you  will  be  asked: 

a  to  generate  a  list  of  tasks  associated  with  operating  the  designated  type  of 
equlpaeat, 

a  to  determine  some  of  the  skills  that  are  Important  in  operating  that  equipment,  and 

a  to  Indicate  the  level  of  each  skill  you  regard  as  necessary  for  successful  perfor¬ 
mance  of  each  listed  task. 

The  following  pages  have  been  designed  to  provide  a  framework  in  which  you  can  cake 
your  responses  conveniently.  A  separate  page  will  be  provided  at  the  conclusion  of  this 
session.  Make  any  consents  you  wish  that  will  laprove  this  survey  farm  Itself,  because 
It  may  be  used  with  other  engineers  In  the  future. 


Aiitnt  that  you  are  part  of  an  engineering  design  teas,  and  that  you  ara  currently 
Involved  in  design  decisions  regarding 


(Enter  system  or  equipment  Identification.) 

think  of  this  particular  subdivision  of  systea  equipment  in  terns  of  your  design 
experience,  and  consider  the  nost  important  tasks  that  will  have  to  be  performed  by 
■llltary  or  civilian  personnel  In  field  operation  of  the  equlpaent. 

Make  a  list  of  these  tasks  la  the  spaces  below.  Please  Indicate  a  nlnloua  of  V  tasks; 
try  to  generate  as  large  a  maker  of  tasks  as  possible.  Develop  your  list  of  tasks  by 
giving  thought  to  the  equipment  in  normal  and  degraded  sodas,  and  In  the  variety  of  environaents 
the  system  would  typically  encounter  In  peacetlae  and  vartlae  circumstances. 


Place  this  page  beside  the  one  on  which  you  wrote  your  list  of  tasks  so  that  you  can 
easily  look  from  one  patje  to  the  other  and  back  a^aln.  The  triads  of  numbers  down  the  left 
margin  are  selected  combinations  of  tasks  from  the  listing  you  generated  on  the  preceedlng 
P««e. 


In  responding  to  the  first  line  below,  look  in  your  list  at  the  three  tasks  corresponding 
to  the  set  of  three  numbers  shown.  Considering  these  three  tasks  only,  think  of  a  skill 
that  is  required  by  any  two  of  the  tasks  hut  not  required  (or  required  to  a  far  lesser 
degree)  by  the  third  task.  You  are  looking  for  a  skill  that  a  military  or  civilian  operator 
or  technician  must  have  for  performing  any  two  of  these  tasks,  but  needs  only  a  minimum 
amount  or  none  of  that  skill  for  the  remaining  task. 

On  the  first  line  toelow,  write  a  name  or  a  brief  descriptive  phrase  for  the  skill  you 
have  deterained  to  be  required  for  tvo  of  the  three  specified  tasks.  Proceed  to  the  second 
line  and  cake  a  similar  Judgment  regarding  the  three  tasks  specified  there.  Continue  until 
you  have  written  a  response  opposite  each  set  of  three  numbers. 

TASK  NUMBER 

FROM  YOUR  LIST 

2.3.4 

1,2,3 

3.4.5 

1.2.5 

1.4.5 


Ksme  of  Skill  Required, 
by  Any  Two  of  the  Tasks 


f 


When  you  have  finished  these  Jud^sents,  give  both  pages  of  your  listings  to  your  Interviewer 

Note:  The  triads  were  specified  If  respondent  had  generated  5  tasks 
on  preceding  page. 


□□□□□□□□□□□□□□□□□□□□ 


the  left  margin  of  thU  page  (and  all  other*  in  this  portion  of  the  survey)  carries  the 
list  of  tasks  you  generated  on  page  3.  At  the  top  of  the  right  margin  you  will  find  one  of 
the  toy  skills  you  listed  on  pegs  3.  In  the  response  space  below,  estimate  the  degree  to 
tdilch  each  task  la  the  list  demands  the  particular  skill  written  In  the  upper  right  corner 
of  the  response  space,  and  write  a  number  for  that  degree  In  the  box  provided.  Use  the 
following  scale  to  select  the  numbers  you  enter  In  the  boxes: 

1  —  does  not  require  this  skill  at  all 


a  —  requires  a  small  amount  of  this  skill 
3  —  requires  a  moderate  amount  of  this  skill 
b  —  requires  a  high  degree  of  this  skill 
5  ~  requires  a  nasi—  smsiat  ef  this  skill 


1 

2 

10 

11 

12 

13 

lk 

13 


When  you  have  entered  a  skill-level  estimate  In  each  of  the  boxes  corresponding  to 
a  task,  proceed  to  the  next  pegs.  It  will  contain  a  reprint  of  the  Instructions  and  the 
task  list,  but  a  different  skill  will  be  written  In  the  upper  right  corner  of  the  response 
spec#.  The  Judgments  required  of  you  are  the  sane  as  those  y«i  have  just  completed  on 
this  page. 


□□□□□□□□□□□□□□□ 


Unstructured  Survey 


ESTIMATES  OF  SKILL  LEVELS 
REQUIRED  BY  MAINTENANCE  TASKS 


This  survey  Is  part  of  a  study  attempting  to  Identify  the  Influence  exerted  on  a  system's 
engineering  design  by  the  designer's  perception  of  the  skills  of  the  persons  who  will 
maintain  the  system. 

On  these  pages  you  will  be  asked: 

e  to  generate  a  list  of  tasks  associated  with  maintaining  the  designated  type  of  equip¬ 
ment, 

a  to  determine  ease  of  the  skills  that  are  Important  In  maintaining  that  equipment',  and 

a  to  Indicate  the  level  of  each  skill  you  regard  as  necessary  for  successful  performance 
of  each  listed  task. 

The  following  pages  have  been  designed  to  provide  a  framework  in  which  you  can  make  your 
reaponsea  conveniently.  A  separate  page  will  be  provided  at  the  conclusion  of  this  session. 
Make  eny  c cements  you  wish  that  will  Improve  this  survey  fora  Itself,  because  It  may  be  used 
with  other  engineers  In  the  future. 


A-7 


Aatu a*  that  you  ara  part  of  aa  engineering  design  team,  and  that  you  are  currently  Involved 
In  design  decisions  regarding 


(Enter  systea  or  equipment  indent  if  icat  ion'.) 

Think  of  this  particular  subdivision  of  system  equipment  in  terms  of  your  design  experi¬ 
ence,  and  consider  the  aost  laportant  tasks  that  will  have  to  be  perforaed  by  n liltary  or 
el'/lllan  personnel  la  field  Maintenance  of  the  equipment. 

Make  a  list  of  these  tasks  In  the  spaces  below.  Please  Indicate  a  minima  of  U  tasks; 
tiy  to  generate  as  large  a  number  of  tasks  as  possible.  Develop  your  list  of  tasks  by  giving 
thought  to  the  equipment  In  normal  and  degraded  modes,  and  In  the  variety  of  environments 
the  systea  would* typically  encounter  In  peacetime  and  wartime  circumstances. 

TASK  LIST 


1 

2 

3 

U 

9 

6 

7 

8 


When  you  have  finished  your  listing,  tell  your  Interviewer  how  many  tasks  you  have 
listed;  he  will  provide  you  with  the  next  page  of  this  survey. 


A-8 


Place  thla  pa**  he  a  Me  the  one  on  which  you  wrote  your  Hat  of  tasks  no  that  you  can 
easily  look  from  or.e  pa:;e  to  the  other  and  burk  a^a'.n.  The  triads  of  misters  down  the  left 
eargln  are  selected  coablnatlon*  of  tasks  from  the  listing  you  generated  on  the  preceedlng 

page. 

In  responding  to  the  first  line  below,  look  In  your  list  at  the  three  tasks  corresponding 
to  the  set  of  three  numbers  shown.  Considering  these  three  tasks  only,  think  of  a  skill 
that  Is  required  by  any  two  of  the  tasks  but  not  required  (or  required  to  a  far  lesser 
degree)  by  the  third  task.  You  are  looking  for  a  skill  that  a  nilltary  or  civilian  operator 
or  technician  oust  have  for  performing  any  two  of  these  tasks,  but  needs  only  a  minimum 
amount  or  none  of  that  skill  for  the  remaining  task. 

On  the  first  line  below,  write  a  naoe  or  a  brief  descriptive  phrase  for  the  skill  you 
have  determined  to  be  required  for  two  of  the  three  specified  tasks.  Proceed  to  the  second 
line  and  sake  a  similar  Ju Assent  regarding  the  three  tasks  specified  there.  Continue  until 
you  have  written  a  response  opposite  each  set  of  three  numbers. 

TASK  NUMBER 
FROM  YOUR  LIST 

2.3.4 

5.6.7 

1.6.7 

1.3.6 

1.3.4 

2.4.5 

2.5.7 

1.2.7 

3.5.7 

3.4.6 

M,5 

2.3.7 

1,2,6 

4,5,6 


When  you  have  finished  these  Ju<l<pent8,  give  both  pages  of  your  listings  to  your  Interviewer 

Note:  These  triads  were  specified  if  responsdent  had  generated  7  tasks 
for  list  on  preceding  page. 


Name  of  Skill  Required 
by  Any  Two  of  the  Tasks  • 


□□□□□□□□□□□□□□□□□□□□ 


lh«  left  ear gin  of  thta  page  (and  all  others  In  this  portion  of  the  survey)  carries  the 
list  of  tasks  you  generated  on  page  2.  At  the  top  of  the  right  margin  you  will  find  one  of 
the  key  skills  you  listed  on  page  3-  In  the  response  space  below,  estimate  the  degree  to 
which  each  task  In  the  list  desands  the  particular  skill  written  In  the  upper  right  corner 
of  the  response  space,  and  write  a  number  for  that  degree  In  the  box  provided.  Use  the 
following  scale  to  select  the  numbers  you  enter  In  the  boxes: 


1  —  does  not  require  this  skill  at  all 

2  —  requires  a  small. amount  of  this  skill 

3  --  requires  a  moderate  amount  of  this  skill 
k  ~  requires  a  high  degree  of  this  skill 

3  —  requires  a  maximum  amount  of  this  skill 

TASK  LIST 


X 

2 

3 

u 

5 

6 

7 

8 
9 

10 

U 

12 

13 

lk 

15 


When  you  have  entered  a  skill-level  estimate  In  each  of  the  boxes  corresponding  to 
a  task,  proceed  to  the  next  page.  It  will  contain  a  reprint  of  the  Instructions  and  the 
task  list,  but  a  different  skill  will  be  written  In  the  upper  right  comer  of  the  response 
space.  The  Judgments  required  of  you  are  the  same  as  those  you  have  Just  completed  on 
this  page. 


□□□□□□□□□□□□□DO 


STRUCTURED  SURVEY 


Instructions  for  Structured  Survey  of  Skill  Level  Estimates 

This  is  a  sample  set  of  cards  similar  to  those  you  will  be  working  with 
for  the  next  part  of  this  study.  Read  through  them  quickly,  just  to  get  an 
idea  of  their  content.  In  making  the  estimates  we  will  ask  of  you,  you  will 
always  have  the  required  tasks  and  skills  before  you  in  this  printed  form. 

(Allow  time  for  subject (s)  to  read  through  cards) 

Each  package/section  of  cards  in  this  survey  contains  a  complete  set  of 
the  task  titles  and  descriptions  you  have  just  read.  In  addition,  the  first 
card  carries  one  of  the  skill  titles/descriptions  we  have  developed,  and  also 
five  cards  comprising  a  rating  scale  for  your  estimates  of  skill  level.  You 
will  make  your  responses  by  sorting  the  task  cards  into  categories,  using  a 
sorting  board. 

From  the. top  of  your  card  deck,  place  the  first  card  in  the  top  center 
bin  on  the  sorting  board.  It  carries  a  particular  skill  title  and  definition, 
and  is  the  skill  we  want  you  to  use  throughout  your  first  sort.  The  next  five 
cards  carry  skill-level  indicators;  place  them  in  the  center  row  on  the  sort¬ 
ing  board  in  sequence  from  left  to  right  (level  #1  in  the  extreme  left  bin, 
level  #2  next,  and  so  on  until  level  #5  is  in  the  extreme  right  bin). 

The  remaining  cards  in  your  first  package  are  a  collection  of  task  titles 
and  descriptions.  Taking  each  task  card  in  succession,  read  its  task  title  and 
description,  note' again  the  skill  title  and  definition  in  the  top  center  bin, 
and  estimate  what  level  of  skill  (from  1  to  5)  is  required  in  performance  of 
the  task  shown  on  the  card  you  are  reading.  Place  the  card  in  the  appropriate 
bin  on  the  bottom  row  of  the  sorting  board,  just  below  the  skill  level  card 
carrying  the  number  of  your  skill-level  estimates.  Continue  through  the 
package  of  task  cards,  sorting  them  according  to  your  best  judgement  among  the 
five  bins  representing  skill-level  estimates.  When  you  have  finished  one 
package,  have  your  interviewer  pick  up  the  cards.  When  the  sorting  board  is 
cleared,  take  the  next  package  of  cards  from  your  deck  and  repeat  the  sorting 
process.  Continue  until  you  have  categorized  all  the  packages  contained  in 
the  deck. 


